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ABSTRACT 

A constrained component analysis method, which bears 
a formal resemblance to the confirmatory factor analysis methods 
developed by K, G. Joreskog (1959) and others, is presented. In 
confirmatory factor analysis, the constraints allow the testing of 
formally structural hypotheses within a model that is falsifiable, 
even in jts "jusu defined" form. In component analysis, the goal is 
to determine whether a component solution that is restricted in 
various ways can still account for an adequate share of the variance 
in the observed data. The consistency and efficiency of the estimates 
under the constrained analysis are considered; however, no 
statistical tests of fit of the component are discussed. In this 
sense, the method is closer to the work in constrained canonical 
correlation by w. s. DeSarbo and others (1982) . Interesting 
applications of constrained component analysis exist in data that are 
longitudinal or cross-sev tional in nature. In these cases, the 
natural constraints may involve the property of being stationary or 
the invariance of the compositing weight, component pattern, or 
component structure matrices. In data measured at a single occasion 
in a single population, constraints might be used to impose sjwple 
structure or achieve a sensible variable clustering. Boundary 
constraints might be used to achieve a positive manifold. (TJH) 
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A paper by Meredith & Hlllsap (1985) presented a view ol component 
analysis in which the regression of observed on component varlabler 
plays the central role. In this regression, the "adequacy" ol a 
component representatlr is measured by the amount ol error in the 
least-squares approximation ol observed by component variables. Let X be 
an nxl vector ol observed random variables, with €(X)=0 and €(Xx')=E. 
To slmpllly discussion, we assume rank(S>n. Let Z=W^X be an mxl 
vector ol component ramdom variables, with W an nxm compositing weight 
matrix to be applied to the observed variables. The paper by Meredith & 
Hlllsap (1985) demonstrated that using a weighted squared error loss 
lunctlon, the optimal component solution is given by the matrix W which 
maximizes the lunctlon 

F(W;2;G) = trtw'lBEUKw'aO"^). (1) 
The matrix G Is an nxn nonslngular matrix that can be used to 
dlllerentlally weight the elements ol X in deriving the components. 
Weighted component analysis is discussed in Mulalk (1972), Meredith 8. 
Mlllsap (1965) gl'"5 an extensive discussion ol possibilities lor 
freighting. 

Note that the value ol (1) cannot exceed tr(GS). A natural 
measure lor the adequacy of a component solution Is the ratio ol (1) to 
triGSll This ratio will be unity 11 m=n, and will approach unity 
lor m<n as the component solution accounts lor a greater share ol the 
observed variance. 

Component Analysis under Unear Constraints 

Belore introducing linear constraints in the component analysis, It 
will be uselul to rewrite (1) m a slightly more general lorm 
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H(U;A.B) = tr{u'AU(u'BUrl> (2) 
with A and B both qxq ipitrlcea ol rank q, and U a qxa matrix. Clearly, 
(1) IS the special case of (2) m whlc^ q=n, arm, A=3GE, and B=S. 

A solution lor U which maximizea H(U;A, B) la found as follows. 
Factor B=LL^ and let A*=L""^AL^^, V^zLV. Then 

H(U;A,B) = H(i;^;A*, I) = tr{iy^ Vl?^(l?^V)-l} (3) 
Let the spectral repreaentatlon of A*=QAq/ a theorem by 
Bellman (1960) gives the matrix which maximizea (3) aa U*=<\iiT« 
where Qq ia an nxm matrix whose columns ire the m eigenvectors 
corresponding to the m largest eigenvalues of A*, and T is any mxm 
nonslngular matrix. The solution identifies a class of matrices U*. 
and we must choose a matrix T to identify a member of thli class. 

In terms of the function F(W;I;G), let TrlL, V/^j:L¥, and 
A^sL^^SGSEL"!. If the spectral representation of A*=QAo/ 
the solution for V^=^nT and W=L*^uT. typically choose T to be 
an Identity matrix or a diagonal matrix of the reciprocal square roots 
of the first m eigenvalues. 

To Introduce the constrained solution to C2), consider 
concatenating the columns of UsCuj, Ug, .... u^l to form a 
column vector V such that sCuJ, ug. .... u^J. We want to 
find a matrix U which maximizes (2) under the constraints 

C V ^ N (4) 
with C a qsxr nonslngular matrix and N an rxl vector. Some or all of 
tht r constraints in (-») may be taken as equalities, but the numb^^r of 
equality constraints may not exceed qs. The constraints in (4) can be 
used to fix elements of U to selected values, to equate elements of U, 
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to aet lower or upper bounds on the elements ol U, or to fix linear 
compositea ol the elementa ol U to aelected values. We can employ aome 
ol the conatralnt5 in (4) lor purpoaea ol identifying U. For thla 
purpoae, a^ equality conatralnta are sufficient. 

Structure and Pattern Hatrlcea 
We began with a description of component analyala that focused on 
the compositing matrix W. Given a solution for W, we can define the 
cooponent pattern matrix P=aW(w'aiO"^ and the component structure 
matrix R=SW. Note that we can write the function H in terms of W, 
P, or R 

H(W;2B5;s) = tr{w'zrsaw(w'ao"^J (S) 
H(P;G,E-1) = tr{P'ca>(PVlp)-lj (6) 
H(R;G.E-1) = tr{R^C«(R'2rlR)-lj (7) 
Clearly, constriants of the form in (4) can be applied to either (6) or 
(7) as well as (5). In other words, we can derive components whose 
pattern or structure matrices conform to specified constraints. 

Extrema cf the Function 
The unconstrained function H(U-,A,B) Is continuous and has 
stationary points, wlfi both a unique global maximum and a unique global 
mlnliDum, assuming distinct eigenvalues for the matrix A*. In 
practice therefore. H(U;A3) is bounded. If we impose m^ equality 
constraints in (4) to identify U, the matrices XJ which satisfy these 
constraints constitute a closed, bounded convex set. Hence there must 
exist both a global maximum and a global minimum for H over this set. 
The addition of further equality or inequality constraints will not 
alter the existence of a global maximum, but the maximum may no longer 
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be unique. In sreneral. 11 additional conatramta are Imposed, the 
maxlQUffl value lor H achieved under theae conatramta will he less than 
that under the -juat identified- solution. 

The Adequacy ol the Conatramed Solution 
We can evaluate the adequacy ol the conatralned component aolution 
In at leaat two waya. An ^'absolute" measure ol adequacy can be 
calculated by replacing the solution lor W. P. or R In (5), (6), or (7) 
respectively. Then either (5). (6), or (7) can be divided by trfGTJ. 
This ratio measures the variance "accounted lor" by the constrained 
solution against the total variance observed. Alternatively, we could 
calulate a "relative" measure ol adequacy by dividing the value ol H 
under the constrained solution by the value ol H under a "Just 
identilied" solution. Here we are measuring tne variance accounted lor 
by the constrained solution against the variance accounted lor by the 
"just identilied" solution. II this ratio is near unity, we can be 
conlident in using the constrained solution in place ol the principal 
component solution. 

Final Remarks 

The constrained component analysis method presented here bears a 
formal resemblance to the conlirmatory lactor analysis methods developed 
by Joreskog (l%9) and others. In conlirmatory lactor analysis, the 
constraints allow us to lormally test structural hypotheses within a 
model that is lalsiliable. even in its "just identilied" form. In 
component analysis, the goal is to determine whether a component 
solution that is restricted in various ways can still account lor an 
adequate share ol the variance in the observed data. We may also be 
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concerned with the consistency and efficiency of the estimates under the 
constrained analysis. But no statistical tests of fit of the component 
-model" are contemplated. In this sense, the method presented here is 
closer to the work In constrained canonical correlation by DeSarbo, 
Hausman, Lin, & Thompson (1982) than to confirmatory factor analysis. 

Interesting applications of constrained component analysis exist In 
data that Is longitudinal or cross-sectional in nature. In these cases, 
the natural constraints may involve statlonarlty or invarlance of the 
compositing weight, component pattern, or component structure matrices. 
In data measured at a single occasion in a single population, 
constraints might be used to impose simple structure or achieve a 
sensible variable clustering. Boundary constrainU might be used to 
achieve a positive manifold. 
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